Flash storage devices are increasing used for storing data in electronic applications. Flash storage devices involve a nonvolatile memory in which data may be electrically erased and programmed often in data blocks of a selected size. Real time lossless compression is widely used in data storage systems, and specifically in flash storage, because it helps to reduce resource usage. For example, in a write flow for a flash storage device, adding compression may provide a number of advantages. These advantages can include a reduction in the total number of writes, thereby enabling a higher effective over provisioning and reducing write amplification. These advantages can also include enabling a larger capacity drive using elastic storage and reduction of power during writes due to reduced writes to the flash storage.
On the other hand, the data storage speed may be decreased substantially because compression processes can consume large amounts of resources. The compression processes often involve substantial and intensive calculations, which correspond to significant consumption of power and time. However, many data blocks may have limited compressibility. More specifically, the resources consumed to compress these data blocks may not be worth expending because little storage space will be saved. In cases when the data is effectively non-compressible, the power and time expended are largely wasted. Thus, a method that can predict a compressibility of data blocks, so the storage system can know whether it would be efficient to compress a given data block and possibly avoid compressing data blocks having low compressibility, might save substantial amounts of resources (e.g., power and latency). Saving these resources can improve the storage system performance.
Some data compressibility prediction algorithms exist based primarily on software implementations. However, for real time storage systems such as flash storage, the compressibility prediction by software may be unrealistic because it is too slow. Also, there are some implementations involving both software and hardware. Such solutions may be faster than software implemented algorithms but they are generally still not fast enough for real time storage systems such as flash storage. In addition, these implementations can be overly complicated.